Repository navigation
docs(contributing): tell contributors how to produce evidence, not just tests - #706
Merged
Merged
Conversation
…st tests
CONTRIBUTING had an "E2E is the evidence layer" rule and no mention of the
spike harnesses, so an outside contributor learned how to prove a FIX and
nothing about how to prove a DIAGNOSIS. That is the half a bug report is
actually made of.
Adds a short section under the same table: the two harnesses (in-process
`spike_*.py` and the hand-driven CDP/HAR capture), when each applies, and
the rule they exist to enforce —
A selector that does not match is evidence about the selector.
It is never evidence about the feature.
Also states the redaction path explicitly, because this is the one place a
contributor is invited to attach a capture: raw HARs carry cookies, Bearer
tokens and prompts, so attach `extract_har_summary.py`'s redacted output,
never the raw file.
Records the platform limitation honestly rather than letting someone
discover it: the CDP harness is PowerShell and Windows-only today, so macOS
and Linux contributors are pointed at the in-process probes or an incident
bundle instead. A portable Python twin is being ported one script at a
time, with `probe_agent_mode.py` as the pattern.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CONTRIBUTING.mdtold contributors how to prove a fix (the E2E evidence layer, #675) and nothing about how to prove a diagnosis — which is what a bug report is actually made of. The spike harnesses became first-class for agents in #704; this makes them findable by people.Adds one section under the existing evidence table:
scripts/dev/spike_*.pyvs the hand-driven CDP/HAR capture inscripts/dev/har-spike/— and which to reach forextract_har_summary.py's redacted summary, never the raw file.*.haris gitignored repo-wide for the same reason.The limitation, stated rather than discovered
The CDP harness is nine PowerShell scripts and Windows-only today (
launch-flow-chrome.ps1alone hardcodes four Windows paths). Rather than let a macOS or Linux contributor find that out the hard way, the section says so and points them at the in-process probes or an incident bundle.A portable Python twin is worth doing — Playwright's
channel="chrome"and_spike_common.resolve_profile_dir()already solve both Windows-bound problems, andprobe_agent_mode.pyis the existing pattern — but incrementally, one script at a time when a spike needs it, not as a 1691-line rewrite nobody can review. Not in this PR.Test plan
check_doc_links.py— all links resolved across 29 files (caught nothing, but thedocs/DEBUGGING.md#automatic-incident-bundlesanchor is real)check_repo_hygiene.py— 1012 tracked files, no violations~/.local/share/gflow-cli/incidents/as the bundle path. The canonical form is
<GFLOW_CLI_HOME>/incidents/, perdocs/CONFIGURATION.mdanddocs/DEBUGGING.md. Corrected before commit.